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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.114, including the 
fee set forth in 37 CFR 1 .17(e), was filed in this application after final rejection. 
Since this application is eligible for continued examination under 37 CFR 1.114, 
and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the 
previous Office action has been withdrawn pursuant to 37 CFR 1 .1 14. 
Applicant's submission filed on 24 March 2010 has been entered. 

Response to Arguments 

Applicant traverses the combination with Rector, making several 
arguments in Applicant's Remarks and in the 132 declaration, both filed 24 March 
2010. However, these arguments are not persuasive. 

Applicant characterizes Rector, noting that the video and signal data may 
be stored as interleaved blocks in an interleaved file format. Applicant then 
argues that the file is not a table relating time slices of a stimulus signal with 
responsive signals as a function of time. Remarks at 9. The examiner disagrees. 
Rector does teach that responsive signals are correlated with time slices of a 
stimulus signal as a function of time. For example, Rector teaches that a 
response signal may be analyzed for an anomaly, and a video frame 
corresponding in time to the anomaly in the response signal is immediately 
retrieved (col. 10, II. 39^13). This example teaches that Rector's data is 
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organized in two dimensions; namely, in a time dimension, and also across 
variables corresponding to a specific time, such that the video frames are 
correlated to the response signals. Therefore, Rector's file format is an 
associative mapping, which may be characterized or illustrated as a table 
because it is organized two-dimensionally. 

Applicant argues that there is no method contemplated or described for 
adding new response variables at a later time to be stored data. Remarks at 9. 
However, these features are not claimed and have no nexus with the claim 
language. 

Applicant argues that Rector's file format requires reading through a 
combined video and data signal to search for values of variables. Remarks at 10. 
The examiner disagrees. The paper cited in the 132 declaration teaches that the 
file format can be easily parsed by reading headers describing the data blocks 
(pg. 156, under "File Format"). Thus, the data blocks themselves do not have to 
be read until the appropriate header is found, and the file can be easily searched 
as described. 

The remaining arguments in the Remarks are moot in view of the newly 
cited art. 

Applicant further attempts to distinguish Rector in the 132 declaration. 
Applicant again attempts to contrast the claimed table and Rector's file format. 
Applicant argues that Rector's file format would require reading through the 
entire signal and video data. Declaration at 2. The examiner again disagrees and 
points to the cited document, which teaches that the file can easily be parsed 
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using header information (pg. 156). Additionally, even if Applicant were correct, 
the claims do not preclude searching through an entire file in order to locate data 
along a "row" or "column" of the mapping. 

Applicant again argues that Rector shows no accommodation for mixing in 
additional data. Declaration at 2. However, as noted above, this feature is not 
claimed. 

Applicant argues that Rector's interleaved file format mixes a plurality of 
data signals into one video file by inserting data into a vertical blanking interval. 
Declaration at 2. This is incorrect; Applicant appears to be confusing Rector's 
analog video storage with Rector's digital IFF storage. Rector teaches that data 
may be stored in the VBI or other spaces within an analog signal on a videotape 
(Fig. 3 and description). Additionally and separately, Rector teaches that data 
may be stored in an interleaved digital format on a mass storage device (Fig. 4 
and description). Further, although Rector does not contemplate a second video 
signal, a second signal could easily be interleaved with the other plurality of 
streams on the mass storage according to the described format. 

For these reasons, Applicant's arguments and declaration are not 
persuasive. 

Priority 

Applicant previously submitted a declaration under 37 CFR 1 .131 
establishing the actual reduction to practice of the invention as being prior to the 
Leroy reference's filing date. 
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However, as previously noted, the evidence does not establish the 
reduction to practice of all the claimed elements. 

Claim 70 describes an apparatus for analyzing responses to a plurality of 
video stimulus streams, comprising storing a multi-channel associative mapping 
containing the plurality of video streams. However, the evidence only discusses 
mapping response data to a single video (e.g. Example C), not to the claimed 
plurality of videos. Thus, the evidence does not establish the reduction to 
practice of claim 70 or its dependents. 

Claim 98 describes associating a stimulus object in a video stream by 
determining whether a stimulus object is present in a particular time slice of the 
video stream. Figure 1 of the evidence appears to show a person in a video 
stream, but no evidence is presented that the software can determine whether or 
not the person is in a particular time slice of the video stream. Claims 99-102 
depend on claim 98 and thus lack support in the same manner. 

Therefore, claims 70, 74, 79, and 98-102 do not benefit from the earlier 
filing date established by the 131 declaration. 



Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 1 02 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 
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Claims 70, 74, and 79 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over US 5,226,177 ("Nickerson") in view of US 5,517,251 
("Rector") and US 6104948 ("Boqart"). 

Claim 70. Nickerson teaches an apparatus for analyzing responses to at 
least one stimulus stream (visually perceptible display, Abstract) comprising an 
input for receiving responses from one or more respondents to the at least one 
stimulus stream (see Abstract, Fig. 1); a correlator including a processor 
configured to correlate the responses with time slices (SMPTE time codes, col. 6, 
II. 48-58) of the stimulus stream to generate an associative mapping of the 
responses and the time slices of the stimulus stream (see Fig. 9 (illustrating a 
mapping of the correlation of the response signal 130 with video time codes); col. 
6, II. 40-66; col. 8, II. 4-7); a storage module coupled with the correlator to store 
the associative mapping including the responses and the time slices of the 
stimulus stream (col. 6, II. 40-43); and a user interface operatively coupled with 
the storage module allowing an operative to search the associative mapping on 
the basis of analyses of the responses (see col. 10, II. 49-56 (discussing various 
statistical analyses that may be performed on the correlated data)). 

Although Nickerson stores the time slices (time codes) as digital signals 
(col. 8, II. 3-7), Nickerson does not further teach storing the stimulus stream as a 
digital signal correlated with the associative mapping; rather, it is stored, e.g., on 
a video tape (col. 7, II. 18-24). Nor does Nickerson explicitly teach a user 
interface allowing an operator to retrieve time slices of the stimulus stream for 
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display wherein playback of the time slices of the stimulus stream may be 
displayed on the user interface. 

Rector teaches an apparatus for correlating a video signal with analog 
response signals (Abstract). The video signal may be stored on a videotape (col. 
2, II. 40-43) and later stored as a digital signal correlated with the response 
signals (col. 2, II. 59-61; col. 4, II. 1-6; col. 7, II. 7-18). The signals are 
associatively mapped by time slices such that the signals are synchronized and 
integrated in a single file (col. 3, II. 11-17). The apparatus includes a user 
interface for searching the associative mapping and retrieving time slices of the 
stimulus stream for display, e.g., in order to display video correlating to 
anomalous response signals (col. 10, II. 36-43; col. 11, II. 5-11 & 40-50). 

It would have been obvious to apply Rector's teaching of digitizing a video 
tape and correlating the digitized signal with response signals to the system of 
Nickerson for the purpose of rapidly and effectively analyzing the correlation 
between the video signal and the response signals (see Rector, col. 1 1 , II. 5-1 1 
& 40-50). 

Nickerson in view of Rector do not teach that multiple video streams are 
stored in the associative mapping. Bogart teaches a similar system for storing 
response data correlated with video data (Abstract). Multiple video streams may 
be correlated (Fig. 2; col. 2, II. 54-56), the video streams including a scene video 
and a subject video (col. 3, II. 30-31). Like in Rector, the stored data may be 
stored both as a videotape and digitally in long term memory for analysis (col. 4, 
II. 35^1). 
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It would have been desirable to correlate responses with multiple video 
streams as taught by Bogart for the purpose of providing a scene view of a 
subject and a subject view, thus storing both broad and detailed stimulus 
information to allow a better analysis of the responses. Since Rector teaches an 
interleaved file format for storing multiple streams, one of ordinary skill would 
have been able to easily store the additional video stream interleaved with the 
other streams. Therefore, the combination would have been obvious. 

Claim 74. Nickerson and Rector further teach storing a multi-channel 
associate mapping (see claim 70 rejection), thus implying a multi-channel 
associative cache (a memory, i.e. cache, for storing the multi-channel associative 
mapping). 

Claim 79. Nickerson further teaches that the stimulus stream may be 
recorded (col. 7, II. 18-24), i.e. delayed in time. 

Claims 80-86, 89, 92, 93, and 95 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Nickerson in view of Rector. 

Claim 80. Nickerson teaches an apparatus for analyzing a response to a 
stimulus (Abstract), the apparatus comprising a stimulus input that receives a 
stimulus signal representing the stimulus (see Fig. 1; col. 2, II. 58-68 (video 
program material is received by the central processor in order to overlay the 
response data on the video signal); Fig. 9); a response input that receives at 
least one response signal (col. 2, II. 58-68; Fig. 9 (illustrating response curve 
130)), each response signal being indicative of a response to the stimulus at a 
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time slice of the received stimulus signal (see Fig. 9 (the horizontal axis of the 
response curve comprises time codes)); and a correlator coupled with the 
stimulus input and the response input, the correlator correlating digitally stored 
time slices of the received stimulus signal with each received response signal as 
a function of time to produce an associative mapping (see Fig. 9 (illustrating a 
correlation 130, i.e. an associative mapping, between the vertical response axis 
and the horizontal time axis); col. 8, II. 3-7 (the time codes are digitally stored by 
the processor)). 

Nickerson does not further teach a storage device having the claimed 
organized structure. 

Rector teaches a storage method for storing response data correlated with 
video data in an interleaved file format such that blocks representing different 
streams of data are interleaved with each other (col. 10 II. 18-31). Because the 
separate variables are all correlated to a time dimension, they can be 
represented by a table of variables. The analysis system allows a single variable 
to be analyzed across time (e.g. a heart rate variable may be analyzed for 
anomalies, col. 10, II. 39^1) and across the variables (e.g., a display frame 
associated with the anomalous heart rate signal can be immediately retrieved, 
col. 10, II. 41-42), thus searching by the "rows" and "columns" of the table to 
retrieve data and video frames. 

It would have been obvious to apply Rector's teaching of digitizing a video 
tape and correlating the digitized signal with response signals to the system of 
Nickerson for the purpose of rapidly and effectively analyzing the correlation 
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between the video signal and the response signals (see Rector, col. 1 1 , II. 5-1 1 
& 40-50). 

Claim 81 . Nickerson in view of Rector further teaches a multi-channel 
associative mapping for recording multiple response signals (Nickerson, col. 8, 1. 
42-55; Rector, Abstract), thus implying a multi-channel associative cache (a 
memory, i.e. cache, for storing the multi-channel associative mapping). 

Claim 82. Nickerson further teaches that each digital time slice comprises 
a frame correlated with a response signal 130, since an SMPTE time code is 
stored on a per-frame basis. 

Claim 83. Nickerson in view of Rector further teaches measuring an 
environmental condition and associating the measurements with the time slices 
in the associative mapping (Rector, col. 6, II. 30-35). 

Claim 84. Nickerson further teaches that the stimulus stream may be 
recorded (col. 7, II. 18-24), i.e. delayed in time. 

Claim 85. Nickerson teaches displaying the data as it received, i.e., 
contemporaneous with its creation (col. 8, 1. 7-11). 

Claim 86. Nickerson teaches that groups of responses may be associated 
with different variables (see Fig. 10); and that viewers may be disposed at 
different angles (e.g., in Fig. 1, a male user of response device 12 is disposed at 
a different angle than a female user of apparatus 1 4). 

Claim 89. Nickerson teaches an analyzer for statistically analyzing the 
response signals (col. 10, 1. 49-56), wherein the analyzer may be used to find 
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selected segments of the stimulus signal, such as a "question 4" segment (Fig. 
10). 

Claim 92. Nickerson in view of Rector further teaches a graphical user 
interface for selecting which statistical analysis is performed (see Nickerson, col. 
8, II. 56-63). 

Claim 93. Nickerson in view of Rector further teaches means for 
displaying the associative mapping (see Nickerson Fig. 9). 

Claim 95. Nickerson in view of Rectorfurther teaches storing the 
associative mapping so that the user can randomly select and access content 
from the storage device (see Rector, col. 11 , II. 5-1 1 & 40-50). 

Claim 90 is rejected under 35 U.S.C. 103(a) as being unpatentable 
over Nickerson in view of Rector and the provided definition of 
"interpolation." 

Claim 90. Nickerson teaches performing a statistical analysis on the 
response signals (col. 10, 1. 49-56), but does not explicitly teach that the 
statistical analysis comprises an interpolation. 

The provided definition 3.b. teaches that the "Method of Interpolation" is 
useful for determining an equation between time and a quantity determined by 
observations. 

Therefore, it would have been obvious to use the described Method of 
Interpolation for the purpose of finding an equation between the time and the 
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observed response signals, thus allowing for the inference of additional 
information based on the known information. 

Claim 91 is rejected under 35 U.S.C. 103(a) as being unpatentable 
over Nickerson in view of Rector and the provided definition of 
"extrapolate." 

Claim 91 . Nickerson teaches performing a statistical analysis on the 
response signals (col. 10, 1. 49-56), but does not explicitly teach that the 
statistical analysis comprises an extrapolation. 

The provided definition 2. a. teaches that extrapolation is useful for 
estimating the values of a series outside a range in which some of its values are 
known. 

Therefore, it would have been obvious to use the extrapolation method for 
the purpose of estimating the values of such series as the response curve 130 
(Fig. 9) or the age groups (Fig. 1 0) outside of the known values in order to infer 
additional information based upon the known information. 

Claims 97 and 103-105 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Nickerson in view of Rector and US 5666157 ("Aviv"). 

Claim 97. Nickerson teaches a method for analyzing responses to at least 
one stimulus stream (visually perceptible display, Abstract) comprising receiving 
responses from one or more respondents to a stimulus stream (see Abstract, Fig. 
1); a correlator for associating the responses with time slices (SMPTE time 
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codes, col. 6, II. 48-58) in which the responses are made (see Fig. 9 (illustrating 
a mapping of the correlation of the response signal 130 with video time codes); 
col. 6, II. 40-66; col. 8, II. 4-7); a storage module coupled with the correlator to 
store the associative mapping including the responses and the time slices of the 
stimulus stream (col. 6, II. 40^3); and a user interface operatively coupled with 
the storage module allowing an operative to search the associative mapping on 
the basis of analyses of the responses (see col. 10, II. 49-56 (discussing various 
statistical analyses that may be performed on the correlated data)). 

Nickerson does not further generating the claimed summary video. 

Rector teaches a similar method for correlating response to video signals 
(Abstract). The apparatus further includes a user interface for searching the 
associative mapping for anomalous responses and retrieving the corresponding 
time slices of the stimulus stream for display (col. 10, II. 36-43; col. 11, II. 5-11 & 
40-50). Rector thus automatically locates "highlights" of the video by analyzing 
the response signal. However, Rector does not further teach compiling the 
highlights into a summary video. 

It would have been obvious to apply Rector's teaching of digitizing a video 
tape and correlating the digitized signal with response signals to the system of 
Nickerson for the purpose of rapidly and effectively analyzing the correlation 
between the video signal and the response signals (see Rector, col. 1 1 , II. 5-1 1 
& 40-50). 

Aviv teaches a security system containing video cameras and audio 
sensors, wherein captured video and audio data is monitored and analyzed (col. 
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10, II. 38-44). When a sensor anomaly is detected, video data is automatically 
stored separately for immediate and subsequent analysis of the video signals 
corresponding to the anomaly (col. 7, II. 26-54). The system thus automatically 
generates a summary video comprising frames related to the anomalous signal. 
The stored video images may be non-contiguous snapshots, e.g. a subset of 
frames sampled for analysis (col. 7, 1. 61 - col. 8, 1. 18). 

It would have been obvious to implement Aviv's automatic summary video 
generation in the method of Nickerson in view of Rector, thus automatically 
separately storing the highlight segments as determined by the automatic 
anomalous signal analysis. 

Claim 103. Nickerson and Rector further teach a multi-channel 
associative mapping for recording multiple response signals (Nickerson, col. 8, 1. 
42-55; Rector, Abstract). 

Claim 104. Nickerson in view of Rector further teaches logging locations of 
stored time slices of the at least one stimulus stream so that the associative 
mapping directs retrieval of time slices for playback (Rector, col. 1 1 , II. 5-1 1 & 
40-50). 

Claim 105. Nickerson in view of Rector further teaches measuring an 
environmental condition and associating the measurements with the time slices 
in the associative mapping (Rector, col. 6, II. 30-35). 
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Claims 98-100 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Nickerson in view of Rector, Aviv, and US 5,812,642 
("Leroy"). 

Claim 98. Nickerson in view of Rector does not further teach determining 
whether an object is present in a time slice and associating the determination 
with the time slice in the associative mapping. 

Leroy teaches a method for analyzing response signals to a promotion 
stimulus stream (Abstract) wherein the stimulus stream comprises objects such 
as people (see Fig. 7: "BOB"; and col. 7, II. 42-60). The method determines 
which time slices the object is present in (see Fig. 7). The determination is 
associating with a time slice in an associative mapping of the stimulus stream, 
the time data, and the response data (see Fig. 7). 

It would have been obvious to add Leroy's method for determining when 
objects are present in the stimulus stream for the purpose of analyzing the 
response data with respect to a particular object, thus providing more specific 
information about the response associated with that object (see Leroy, col. 7, II. 
50-52). 

Claim 99. Leroy further teaches that the object comprises a person (col. 7, 
II. 42-60). 

Claim 100. Nickerson further teaches that the stimulus stream is a video 
(col. 7, 1. 59-68), which comprises an audio stream. 
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Claims 101 and 102 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Nickerson in view of Rector, Leroy, Aviv, and Cobbley. 

Claims 101 , 102. Nickerson in view of Rector and Leroy does not further 
teach an automatic audio analyzer for analyzing the audio stream to produce text 
strings that are searchable through the user interface. 

Cobbley teaches an apparatus for selectively playing back segments of a 
video by selecting index information associated with the segments (Abstract). 
The user may search for text strings to locate the selectable segment (col. 10, II. 
21-38; col. 12, II. 41-56). The text strings may be generated by an automatic 
audio analyzer (col. 4, II. 41-43). 

It is obvious to combine known elements according to known methods to 
yield predictable results. Therefore, it would have been obvious to combine 
Cobbley's automatic audio analyzer with the apparatus of Nickerson in view of 
Rector and Leroy according to the described method of searching for and 
selecting video segments using text strings, thus yielding predictable results 
because the apparatus divides videos into selectable segments (time slices). 

Conclusion 

Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Bennett Ingvoldstad whose telephone number 
is (571) 270-3431 . The examiner can normally be reached on M-F 9-5 EST. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Scott Beliveau can be reached on (571) 272-7343. The 
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fax phone number for the organization where this application or proceeding is 
assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786- 
91 99 (IN USA OR CANADA) or 571-272-1000. 



/Bennett Ingvoldstad/ 
Examiner, Art Unit 2427 

/Scott Beliveau/ 

Supervisory Patent Examiner, Art Unit 2427 



